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Abstract 


This  report  details  a  brief  comparison  that  was  made  between  two  software  packages,  XJTech’s 
AnyLogic  and  Aptima’s  Distributed  Dynamic  Decision-making,  for  the  purpose  of  evaluating 
their  suitability  to  microworld  research  related  to  the  training  of  command  decision-making. 


Resume 


Le  present  rapport  presente  en  detail  une  breve  comparaison  de  deux  logiciels,  Anylogic  de 
XJTech  et  Distributed  Dynamic  Decision-making  d’Aptima,  qui  a  ete  effectuee  pour  evaluer  leur 
pertinence  comme  outils  de  recherche  sur  les  micro-mondes  en  vue  de  la  formation  en  prise  de 
decision  de  commandement. 
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Executive  summary 


Evaluation  of  Simulation  Platforms  for  Training  of  Command 
Decision  Making:  Final  Report 

Gerard  Torenvliet;  lain  Culligan;  DRDC  Toronto  CR  2008-059;  Defence  R&D 
Canada  -  Toronto;  March  2008. 

Introduction  or  background:  This  report  details  a  brief  comparison  that  was  made  between  two 
software  packages,  XJTech’s  AnyLogic  and  Aptima’s  Distributed  Dynamic  Decision-making 
(DDD),  for  the  purpose  of  evaluating  their  suitability  to  microworld  research  related  to  the 
training  of  command  decision-making. 

Results:  The  results  of  our  comparison  indicate  that  XJTech’s  AnyLogic  is  a  software  package 
that  is  compatible  with  microworlds  research  (e.g.,  Gonzalez,  Vanyukov,  &  Martin,  2005).  We 
also  found  that  Aptima’s  DDD  platform  could  be  compatible  with  microworlds  research,  but  that 
this  tool  typically  provides  a  decision-making  environment  that  is  of  a  higher  fidelity  than  is 
typically  associated  with  the  authors’  understanding  of  microworlds  research. 

Significance:  This  comparison  can  help  Defence  Research  Development  Canada  -  Toronto  to 
better  understand  the  software  tools  to  be  used  in  the  context  of  Project  12sk  (Accelerated 
Mission-based  Training). 

Future  plans:  If  further  comparison  is  required,  this  research  is  likely  best  extended  by 
performing  a  more  detailed  and  specific  application-oriented  test  of  both  tools. 
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Gerard  Torenvliet  et  lain  Culligan;  RDDC  Toronto  CR  AD;  R  &  D  pour  la 
defense  Canada  -  Toronto;  Mars  2008 

Introduction  ou  contexte  :  Le  present  rapport  presente  en  detail  une  breve  comparaison  de  deux 
logiciels,  Anylogic  de  XJTech  et  Distributed  Dynamic  Decision-making  (DDD)  d’Aptima,  qui  a 
ete  effectuee  pour  evaluer  leur  pertinence  comme  outils  de  recherche  sur  les  micro-mondes  en 
vue  de  la  formation  en  prise  de  decision  de  commandement. 

Resultats  :  Selon  notre  comparaison,  Anylogic  de  XJTech  est  un  logiciel  compatible  avec  la 
recherche  sur  les  micro-mondes  (voir  Gonzalez,  Vanyukov  et  Martin,  2005).  Nous  avons  aussi 
trouve  que  la  plateforme  DDD  d’Aptima  pourrait  etre  compatible  avec  la  recherche  sur  les  micro- 
mondes,  mais  que  cet  outil  foumit  un  environnement  de  prise  de  decision  plus  fidele  que  ce  que 
qui  est  habituellement  associe  a  la  recherche  sur  les  micro-mondes  telle  que  comprise  par  les 
auteurs. 

Importance  :  Cette  comparaison  peut  aider  Recherche  et  developpement  pour  la  defense  Canada 
-  Toronto  de  mieux  comprendre  les  outils  logiciels  destines  a  etre  utilises  dans  le  contexte  du 
projet  12sk  (Formation  acceleree  en  prise  de  decision  dynamique  pour  le  commandement). 

Plans  futurs  :  Si  d’autres  comparaisons  sont  requises,  la  presente  recherche  sera 
vraisemblablement  mieux  poursuivie  en  effectuant  des  essais  plus  detailles  et  plus  specifiquement 
axes  sur  les  applications  de  ces  deux  outils. 
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1  Introduction 


Military  commanders  are  increasingly  being  called  upon  to  make  decisions  in  complex,  dynamic 
environments,  where  predicting  and  managing  higher-order  effects,  making  sense  of  large 
amounts  of  data  from  varied  sources,  and  working  with  non-military  partners  with  in  novel 
cultural  settings  are  becoming  the  norm.  These  challenges  are  most  salient  at  the  operational  and 
strategic  levels  of  military  operations.  However,  current  training  of  commanders  in  the  Canadian 
Forces  (as  well  as  those  of  other  militaries)  focuses  on  tactical-level  decision  making,  and  does 
not  fully  address  the  complexity  of  decision  making  at  higher  levels.  The  use  of  microworlds, 
dynamic  and  interactive  simulations  of  decision-making  environments,  shows  promise  to  assist  in 
training  for  dynamic  decision  making  in  complex  systems,  by  capturing  the  essential  human 
cognitive  and  decision-making  elements  of  those  systems  in  a  simple  and  tractable  model. 
Defence  Research  and  Development  Canada  (DRDC)  Toronto  is  currently  performing  research 
(Project  12sk,  Accelerated  mission-specific  training)  to  test  the  assumptions  behind  the  use  of 
microworlds  in  training,  and  to  develop  techniques  for  the  development  of  microworlds  for 
training. 

This  report  documents  a  high-level  comparison  made  between  two  software  tools  for  the 
development  of  microworld  simulations,  XJTech’s  AnyLogic  and  Aptima’s  Distributed  Dynamic 
Decision-making  (DDD),  to  determine  their  applicability  to  this  research. 
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2  Method 


The  comparison  between  XJTech’s  AnyLogic  and  Aptima’s  DDD  documented  in  this  report  was 
conducted  using  AnyLogic  Advanced  6.2.1  and  DDD  4.0  Service  Pack  4  (which  includes  the 
Visual  Scene  Generator  4.0).  The  authors  acquired  the  software  and  installed  it  on  a  typical  laptop 
running  Microsoft  Windows  XP  Service  Pack  2. 

The  authors  reviewed  the  product  documentation  and  the  demonstrations  packaged  with  each 
tool,  and  then  performed  a  comparison  of  the  two  tools  based  on  the  following  criteria: 

•  the  capability  and  flexibility  of  the  modelling  languages  to  capture  processes  and  dynamics 
at  a  level  relevant  to  microworld  research; 

•  the  provision  of  debugging  utilities  to  allow  modellers  to  troubleshoot  and  resolve  problems 
with  model  logic; 

•  the  capability  and  flexibility  of  the  simulation  environments  to  provide  user  interfaces 
relevant  to  experimental  investigations  of  human  cognition  and  decision-making  in 
microworld  research; 

•  capability  to  develop  distributed  simulations  where  different  users  interact  with  the  same 
model  simultaneously,  and  communicate  with  each  other  via  chat/text  messages; 

•  the  capability  and  flexibility  of  the  modelling  languages  and  simulation  environments  to 
allow  for  data  collection  pertaining  to  both  run-time  system  states  and  user  interface 
manipulations; 

•  the  potential  for  the  functionality  of  the  simulation  environments  to  be  extended  through 
third-party  user  interfaces  and  data  collection  and  reduction  applications;  and, 

•  the  overall  usability  of  the  simulation  environments  for  research  projects  in  which  the  model 
will  be  provided  and  maintained  by  a  lead  researcher  (e.g.,  the  DRDC  Scientific  Authority) 
with  user-interfaces  and  data  collection  to  be  designed  and  provided  by  a  third-party 
contractor. 

The  results  of  this  comparison  are  documented  in  Section  3,  below. 
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3  Results 


3.1  General 

This  section  details  the  results  of  the  comparison  performed  between  XJTech’s  AnyLogic  and 
Aptima’s  DDD.  The  comparison  included  all  of  the  criteria  included  listed  in  Section  2  (above)  as 
well  as  other  criteria  deemed  notable  by  the  operators. 


3.2  Comparison  on  Main  Criteria 


The  table  below  details  a  comparison  between  AnyLogic  and  DDD  on  the  main  criteria 
introduced  in  Section  2  (above). 


Criteria 

Comparison  Details 

Capability  and  flexibility  of  the 
modelling  languages  to 
capture  processes  and 
dynamics  at  a  level  relevant  to 
microworld  research. 

Both  AnyLogic  and  DDD  can  be  considered  to  be  tools  for  the 
development  of  microworld  simulations  in  that  they  allow  for 
capturing  a  simplified  set  of  system  dynamics  for  the  purpose  of 
research.  The  chief  difference  between  them  is  their  intended 
purpose.  DDD  is  a  specialized  tool  for  the  purpose  of 
conducting  experimentation  on  military  decision  making  in  a 
tactical  (or  potentially,  operational)  environment,  while  AnyLogic 
is  a  tool  for  developing  simulations  of  dynamic  processes. 

AnyLogic  does  have  the  flexibility  to  capture  a  large  range  of 
processes  and  dynamics  at  a  level  relevant  to  microworld 
research,  and  its  pre-packaged  libraries  can  be  extended  by 
someone  knowledgeable  of  the  Java  programming  language. 
DDD  can  capture  a  large  range  of  processes  and  dynamics,  but 
these  seem  to  be  tailored  to  the  geo-spatial  evolution  of  a 
tactical  or  operational  combat  situation.  DDD  could  be  extended 
by  someone  knowledgeable  of  the  C#  programming  language. 

Provision  of  debugging  utilities 
to  allow  modellers  to 
troubleshoot  and  resolve 
problems  with  model  logic. 

Neither  of  these  tools  includes  a  debugging  environment  to 
troubleshoot  problems  in  the  semantics  of  their  models. 

There  is  some  support  for  troubleshooting  syntax  problems  in 
each  tool,  and  better  support  for  this  task  can  be  achieved 
within  a  software  development  environment.  Debugging  the 
syntax  of  DDD  models  will  likely  be  more  difficult  than  for 
AnyLogic  models,  because  DDD  models  are  typically  developed 
in  a  client/server  architecture,  so  there  are  more  potential 
causes  for  errors. 

In  either  tool,  facility  with  the  programming  language  on  which 
the  tool  is  based  (Java  for  AnyLogic  or  C#  for  DDD)  will  likely  be 
of  benefit  in  debugging  tasks. 

DRDC  Toronto  CR  TBD2008-059 


3 


Criteria 

Comparison  Details 

Capability  and  flexibility  of  the 
simulation  environments  to 
provide  user  interfaces 
relevant  to  experimental 
investigations  of  human 
cognition  and  decision-making 
in  microworld  research. 

AnyLogic  models  can  be  provided  with  simple  user  interfaces 
that  use  most  standard  Windows  controls  (e.g.,  push  buttons, 
check  boxes,  text  fields,  etc.).  More  advanced  user  interfaces 
must  be  provided  via  a  Java  wrapper  application. 

DDD  models  have  a  default  map-based  user  interface,  which 
seems  appropriate  to  many  investigations  involving  team  work 
in  team  work  in  tactical  situations.  Custom  user  interfaces  can 
be  developed  in  C#,  from  which  you  can  access  the  DLLs  via 
the  DDD  API. 

Capability  to  develop 
distributed  simulations  where 
different  users  interact  with  the 
same  model  simultaneously, 
and  communicate  with  each 
other  via  chat/text  messages. 

AnyLogic  has  not  been  developed  to  be  a  distributed  platform, 
but  instead  has  been  developed  to  allow  for  the  modelling  and 
simulation  of  system  dynamics  on  a  single  platform  by  a  single 
user.  AnyLogic  could  likely  be  extended  by  the  provision  of 
custom  client/server  components,  but  this  is  not  the  designed 
use  of  AnyLogic  models. 

DDD,  on  the  other  hand,  has  been  designed  from  the  ground  up 
to  be  a  distributed  experimentation  platform  and  so  has  been 
developed  on  a  client/server  model.  Distributing  models  to 
clients  is  easy  (each  time  users  log  into  the  server  web  page,  all 
appropriate  models  are  updated  and  readied  for  use),  and  the 
out-of-the-box  experimental  interface  includes  text  chat  with  the 
capability  for  grouping  operators  using  nets. 

Capability  and  flexibility  of  the 
modelling  languages  and 
simulation  environments  to 
allow  for  data  collection 
pertaining  to  both  run-time 
system  states  and  user 
interface  manipulations. 

Both  AnyLogic  and  DDD  allow  for  collecting  data  about  the 
model  states  as  they  develop  over  time.  DDD  seems  to  include 
more  out-of-the-box  functionality  for  collecting  data  about  user 
interface  manipulations. 

Potential  for  the  functionality  of 
the  simulation  environments  to 
be  extended  through  third- 
party  user  interfaces  and  data 
collection  and  reduction 
applications. 

Both  tools  can  be  extended  using  their  respective  base 
programming  languages  to  allow  for  the  collection  of  complex 
user-interface  manipulation  data  and  data  reduction.  The  ease 
of  extending  depends  on  the  user’s  facility  with  these 
programming  languages. 

Overall  usability  of  the 
simulation  environments  for 
research  projects  in  which  the 
model  will  be  provided  and 
maintained  by  a  lead 
researcher  (e.g.,  the  DRDC 
Scientific  Authority)  with  user- 
interfaces  and  data  collection 
to  be  designed  and  provided 
by  a  third-party  contractor 

It  is  the  opinion  of  the  authors  that  the  AnyLogic  tool  is  best 
suited  to  a  research  structure  in  which  the  model  is  provided 
and  maintained  by  a  lead  research  and  user  interfaces  and  data 
collection  are  designed  by  a  third-party  contractor.  This  is 
because  AnyLogic  models  seem  to  be  more  easily  separable 
from  the  interfaces  which  operate  them. 

DDD  could  also  be  used  in  an  environment  where  the  model  is 
provided  and  maintained  by  a  lead  researcher,  but  in  this  case 
the  model  repository  from  the  lead  researcher’s  point  of  view 
would  likely  be  a  model  requirements  document  /  specification. 
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3.3  Other  observations 


In  addition  to  the  comparisons  based  on  the  main  criteria,  as  documented  in  Section  3.2  (above), 
the  authors  also  noted  the  following  points  of  comparison: 


Criteria 

Comparison  Details 

Installation 

The  installation  of  AnyLogic  is  simple  and  straightforward. 

On  a  clean  machine,  the  installation  of  DDD  is  similarly 
straightforward  provided  the  steps  in  the  documentation  are 
followed.  However,  since  DDD  rests  on  a  client/server 
architecture,  there  are  installation  dependencies  that  can  be 
difficult  to  resolve. 

Support 

AnyLogic  support  is  provided  via  email,  and  typically  involves  a 
24h  response  cycle. 

DDD  support  is  very  quick  and  responsive,  and  is  provided  by 
phone  or  email. 

Licensing 

AnyLogic  licenses  must  be  activated  over  the  internet  and  are 
linked  to  a  specific  computer  (presumably  by  MAC  address).  In 
the  DND  environment  it  will  likely  require  IT  assistance  to 
properly  configure  the  firewall  to  allow  for  activation. 

DDD  licensing  is  provided  via  a  standard  license  code.  There 
are  no  restrictions  on  moving  licenses  from  computer  to 
computer,  provided  the  total  number  of  seats  in  a  given  license 
is  not  exceeded.  A  single  server  license  comes  with  four  client 
licenses;  any  additional  client  licenses  must  be  purchased  for 
an  additional  fee. 
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4  Conclusions 


Based  on  our  review  of  AnyLogic  and  DDD,  it  is  our  opinion  that  while  both  tools  are  suitable  for 
microworlds  research,  AnyLogic  is  most  compatible  with  the  modelling  techniques  currently 
being  pursued  in  this  research  stream  (e.g.,  Gonzalez  et  al.,  2005).  It  allows  for  building  models 
with  simplified  characteristics  and  dynamics  that  may  be  able  to  capture  the  cognitively  relevant 
properties  of  a  broad  range  of  decision-making  environments.  DDD  may  also  be  useful  for  this 
purpose,  but  the  results  of  our  brief  review  indicate  that  DDD  is  tailored  to  making  decisions  in  a 
map-based  geo-spatial  context.  This  environment  is  of  a  higher  fidelity  than  is  typically 
associated  with  the  authors’  understanding  of  microworlds  research. 

However,  our  review  indicates  that  if  the  research  requirement  is  to  investigate  or  train  team- 
based  decision  making,  DDD  may  be  the  more  appropriate  tool.  It  includes  a  robust  client/server 
architecture  and  has  been  developed  for  this  purpose.  AnyLogic  would  require  significant 
modification  and  extension  to  scale  to  a  client/server  implementation. 
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